Douglas-Rachford Splitting for Cardinality Constrained Quadratic Programming

In this report, we study the class of Cardinality Constrained Quadratic Programs (CCQP), problems with (not necessarily convex) quadratic objective and cardinality constraints. Many practical problems of importance can be formulated as CCQPs. Examples include sparse principal component analysis [1], [2], cardinality constrained mean-variance portfolio selection problem [3]–[5], subset selection problem in regression analysis [6], and many more. Since these problems are NP-hard in general, branch-and-bound type methods are usually used to solve them. In [7], a tailored branch-and-bound implementation with pivoting algorithm is introduced. In [8], a local relaxation algorithm is discussed which is based on sequence of small, local, quadratic-programs. In this report we focus on approximately solving CCQPs using Douglas-Rachford (DR) splitting. Douglas-Rachford splitting is an operator splitting method which was introduced in 1970s, but attracted a lot of attention after its close relation with alternating direction method of multipliers was shown [9]. Even though it is originally designed to solve convex optimization problems, recently, using ideas from semi analytic geometry [10], some convergence results for the noncovex case was shown [11]–[13]. Also this method shows competitive performance in practice. The rest of this report is as follows: in §2 we introduce a general framework for CCQPs that we study. In §3 and §4 we discuss two important examples, finding a sparse solution to a set of linear equations, and sparse principal component analysis (SPCA). In §5 discuss future work and §6, Appendix, includes some of the proofs that we omitted from this report.